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(54) Data transmission system and method 

(57) A data transmission system for transmitting an 
encoded stream via a network, has a video transmis- 
sion unit having a stream coding unit (34) tor generating 
an encoded stream which is packetoedbi units of aban- 
donabio data, and in which a header Including a packet 
WenSier also serving as packet priority is added to each 
pacta, and a stream shaping processing unit (02,12) 



for determining transmission or abandonment of each 
packet In the encoded stream generated by the stream 
coding unit (34) using the packet Identifier Included bi 
the header of each packet in accordance with the desig- 
nated bit rate. 
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Description 

The present invention relates to a data transmission 
system and method, which process roaMirno data such 
as video data, audio data, and the Oka as packet stream 
data. 

In recent years, along with the development of the 
computer network techniques and the comnuirBcation 
techniques of digital Mbr maflo a a date transfer system 
for transferring a time-serial data sequence (to be 
referred to as stream data hereinafter) to realize. e.g., 
video on demand, has been developed 

In such data transfer system, a video or audio sig- 
nal ot e.g., a requested movie is converted Into dgBa) 
information to form stream data, which is transmitted 
from a transmitting system to a receiving system via a 
network. 

In general. In order to distribute delta such as video 
data Including audio data via a network, moving Image 
coding such as MPEG (Moving Picture Experts Group), 
MotforvJPEG [Joint Photographic Coding Experts 
Group), H.281, or the like is used, and such data Is 
transmitted as an encoded packet stream 

When murtimeda data such as audio data, video 
data, and the like is transferred in real-time on a network 
shared by a plurality of users, since the required net- 
work bandwidth cannot always be assured, the bit rate 
of an encoded video stream is reduced in correspond- 
arcs with the avaflabJs network bandwidth upon trans- 
ferring the stream. 

In one conventional bit rate reduction method, cod- 
ing processing and stream transfer are perform ed paral- 
lel to each other, and when the bit rate control Is 
required, an encoder is fesdbaofccontiolled to adjust 
the cooing parameters (imago quality, the number of 
frames, and the 6ke). However, thte method cannot be 
appQed to an cn -demand video communication system 
that transmits pre-stored streams as encoded packet 
streams since the encoder end data trans mi ssion unit 
depend on each other in the system arrangement. 

In the following description, a system In which an 
encode unit and a data transfer unit are independent 
from each other, and the data transfer unit reduces the 
bit rata of an encoded stream will be exempted. Note 
that reducing the bit rate of an encoded stream to corre- 
sponded that avalable network bandwidth wil be 
referred to as stream shaping processing hereinafter. 

tn general three following stream shaping memods 
(1) to (3) for video streams are avalable: 

(1) To reduce the number of display frames (time 
resolution). 

(2) To reduce the Image sfcre fwWKh, height) (spatial 
resolution). 

(S) lb reduce the number of bits par pixel of an orig- 
inal image (the number o! gradation levels or color 
resolution). 



For example, when the time resolution of 8 stream 
(ao., Moton-JPEG) which has been intraframe- 
encoded at 30 mimes/sac is hafved by the method of 
reducing the number of display frames to transmit a 

8 stream at 15 frames/sec, the boundajy ol erxxried data 
frame data is detected by stream analysis during the 
data transmission proce ss ing, and data are alternately 
transmitted and abandoned for every other frames. 
On the other hand, when the method of reductog 

10 the Image size (width, height), I.e., the method of chang- 
ing the spatial resolution is used, the individual frame 
data In a stream must be encoded while being sepa- 
rated into low-resolution data and high-resolution 
expanded data for compensating the low-resofuiJon 

15 data (i.e., a rierarchicafly encoded stream), and the bit 
rate is tBdisced by abandoning high -resolution 
expanded data upon data transfer. In this case, stream 
shaping as a combination of the spatial resolution or 
time resolution can be attained. 

20 The term "testability" in moving image coding 
means that two or mere images having different spatial 
resolutions and time resolutions can be decoded from a 
single bftstream The above-mentioned bit rate control 
utilizes the scalehflfty of an encoded stream. 

25 MPEG as the international standards of moving 
image coding describes an encoded stream having a 
scalability function (to be referred to as a scalabfy 
encoded stream hereinafter), but such stream realizes 
the scalabltrty In a decoder and doss not take scalabflrty 

so upon stream transmission into consideration. 

For example, MPEG defines the multiplexing 
method of video and audio data. However, it is riot easy 
to separata a data portion corresponding to a epec&ic 
picture (eg.. B picture) In a video stream from a mufti- 

35 pf exed stream. 

As for a method that handles an MPEG video 
stream, a method of setting a ptorafty of fevers in corre- 
spondence with the picture types is known. This method 
designates one of all data cfistribute level, B picture 

40 abandon level. B. P picture abandon level, and an video 
abandon level (distribute audio data alone); it allows 
only discrete rate control. 

tn a muHplBxed stream of video and audio data, tha 
rate control for preferentially transmitting audio data 

45 whose quality may deteriorate upon sub-sampfing or 
decimation, and starting to reduce the bit rate from a 
video stream is known. However, the importances of 
video and audio data differ deperxfrnj on the video con- 
tents and user's requirements. Also, since a stream that 

so muftfrfaxBB a plurality of video data must often be proc- 
essed, it is a very serious restraint for the user rf he or 
she cannot set the policy of the bit rate control 

If the bit rate control In a relay node or multicast 
video distribution is taken into consideration, the bit rate 

55 control method Is preferably independent from the cod- 
ing schema For this purpose, a stream structure used 
InlhBbit rate control method must be easily expanded 
from typical moving Image coding schemes such as 
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MPEG. Motion-JPEG, K261 , and the Eka, and must be 
suitable tor the bit rate control after coding. However, 
since trie conventional method defines a spedal stream 
structure format depending on the moving Image ooolng 
schama used, and performs the bft rate control using 
the defined stream, the expanoVabifity of the stream 
structure Is limited. 

The present invention has been made in considera- 
ticn di the above situation, and has as te object to pro- 
vide a data transmission system and method, writer can 
realize dynamic bit rale control corresponding to the 
available network bandwidth Independently of the mov- 
ing image cooing scheme used. 

According to the present Invention, there Is pro- 
vided a data transmission system comprising: 

Input means far Inputting an encoded stream which 
is packetized in units of abandonable data, and in 
which a header including a packet Identifier Is 
added to each of packets; 
transmission means tor transmitting the encoded 
stream Input from the frpui means onto an network; 
designation means for designating a bit rate; and 
control means tor controlling a bit rate upon trans- 
mission of the encoded stream by the transmission 
means by abandoning a specific packet usfrig 
packet priority determined on the basis of the 
packet identffier included in the header of each 
packet In accordance with the bit rate designated by 
the designation means. 

With this arrangement, since stream coding means 
can generate an encoded stream by simply peckstizjng 
encoded data In unite of abandonable data Independ- 
ently of the cooing scheme itself, the present invention 
can be applied to a system using any of arbitrary scala- 
ble cooing schemes such as MPEG, Motion -JPEQ, 
H.261, and the Do. That Is* the present Invention 
defines only the pack&tizing method of encoded data. 
Such packatuang method can be easily expanded from 
existing cooing schemes, and the compatibility of 
stream transmission/reception means having a bit rate 
control function can be Improved. Also, since the bft rate 
control is attained by looking up information atone in the 
packet header without performing any complex stream 
analysis during stream transmission processing, high- 
speed proces sin g can be reafzed. 

"The stream shaping processing means comprises 
table generation means for generating a f Storing infor- 
mation labte In which a correspondence between the 
packet identifier included in the header of each packet 
of the encoded stream generated by the stream coding 
means, and packet priority is registered, and f Qtertig 
discrimination means for discriminating transmission or 
abandonment of each of the packets with reference to 
the filtering information table generated by the table 
generation means on the basis of the packet identifier 
Included in me header extracted from each pacta* tn the 



encoded stream to be transmitted. 

With this arrangement, the filtering Information 
table is automatically generated on the basis of packet 
identifiers which are tocfcded in headers added by 

ff stream coding means and also serve as packet priority 
levels, and the bit rate control is done using this table. 
More specifically, since me system comprises the table 
generation means, the packet identifier can be pre- 
vented from being uniquely defined as packet priority. 

io The table generation means sets the correspond- 
ence between the packet tterttifier and packet priority in 
accordance wftrt an externally Input designation. 

With this arrangement the correspondence 
between the packet Uentffiers and packet priority levels 

75 registered in the filtering information table is changed 
externally, i.e.. In accordance with user's designations. 
Therefore, the bit rate control can be made In accord- 
ance with user* designations. 

The stream ooolng means generates an encoded 

20 stream by cycticaiy inserting control packets indicating 
coding cydes, and the stream shaping processing 
means determines transmission or abandonment of 
each packet not to exceed an evaBabie network band- 
wkffii (or the designated bit rate) whfle Increasing the 

25 number of packets to be transmitted in unite of cycles 
discriminated based on the control packets. 

With this arrangement, even when the available 
network bandwidth changes during transmission of an 
encoded stream. It te checked at cydes discriminated 

so based on the control packets H the current bit rate corre- 
sponds to the available network bandwidth, and control 
is made to sequentially and preferentially transmit pack- 
ets in the order from those with higher priority levels 
based on the packet priority levels within the allowable 

ss range, thus dynamically coping with changes in network 
bandwkfth. 

The stream cooing means provides a hierarchical 
structure to the packet identifier Included In the header 
to express packet priority based on the contents of the 
40 individual layers of the hierarchy. 

With this arrangements since the packet Identifier 
can have a hierarchical structure to represent packet 
priority levels based on the contents of the Individual 
layers of the hierarchy; when the encoded stream to be 
4s transmmed Is a multiplexed stream, the packet priority 
levels can be finery set for example, priority levels can 
be assigned to the individual multiplexed data. 

The stream cooing means allows to Discriminate if 
the packet identifier included in the header has absolute 
so packet priority or variable packet priority. 

With this arrangement the presence of absolute 
packet priority that can not be changed is clearly indi- 
cated. For example, If MPEG Is used as the coding 
scheme, an encoded stream have abscfute priorities in 
ss order of I picture, P picture, and B picture. 

The stream cooing means can set the packet fcfen- 
tfler included m the header to kxScate whether or not a 
packet Is abandonable, 
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Wrfri this arrangement, when the quality of decoded 
data is consxieraJbly impaired or dec oded data becomes 
nonsensical data if a packet in the middle of a stream Is 
abandoned like in a stream of audio data which is not 
hierarchically encoded, user data, and the Eta, it Is 5 
clearly indented that all the packets are to be transmit- 
ted without parmittfcig any local packet abandonment or 
are not to be transmitted stall. 

A relay node which Is arranged on a network route 
and Is boated between a data trarsmlfflng system and 10 
a data receiving system, comprises reception means for 
receiving an encoded stream from said transmitting sys- 
tem and stream shaping processing means for deter- 
mfnfcg transmission or abandonment of each of packets 
in the encoded stream which is pacteb'zed in units of 75 
abandoned! 0 data* in which a header snchxfing a packet 
Wantrflar also serving as packet priority Is added to each 
of the packets, and which is transmitted from fre data 
transmitting system, using priority based on the packet 
ktenffier included in the header of each of the packets 20 
In units of packets In accordance with an available net- 
work bandwidth, and transmission means tor transmit- 
ting the encoded stream made up of the packets which 
are determined to be transmitted by the stream shaping 
proce ss ing means to the data receiving system via the ss 
network 

With this arrangement, even in the relay node 
located between the data transmitting system and data 
receiving system on the network route, stream shaping 
pr oce ssin g corresponding to the network state (availa- ao 
We network bandwidth) to the data racervtag system 
can be performed in the same manner as tat in the 
data transmitting system. That is, the stream shaping 
processing can be performed for an encoded stream In 
multiple stapes. 35 

Additional objects and advantages of the present 
invention will be set torth in the description which fol- 
lows, and h part wa be obvious from the description, or 
may be learned by practice of the present invention. 

The objects and advantages of the present invert- 40 
tkm may be realized and obtained by means of the 
instrumentalities and combinations particularly pointed 
out In the appended claims. 

This invention can be more fufly understood from 
the following detaled description when taken In con- 45 
junction wfth 1he accompanying drawings* In which: 

FK3L 1 tea block diagram for explaining the outline 
of a data transmission system according to the 
present Invention; 50 
RG. 2 is a block diagram showing the arrangement 
of a video transmission system in an embocfiment 
of the present bivention; 

FtG. 3 is a Weekday ram stowing the arrangement 
of a video communication system Inducing the ss 
video tran smissio n system shown in FIG. 2 in the 
embodiment of the presort invention; 
FIG. 4 Is a block diagram showing the arrangement 



of a relay node having a stream shaping function in 
the embodiment of the present invention; 
FK3. 5 is a view for explaining the structure of an 
encoded stream used in the embodiment of the 
present Invention; 

RG. 6A rs a view tor explaining the packet format 
(packet header) In me embodiment of the present 
invention; 

RG. 68 shows example (1) of a data packet In the 

ernbocflmert of the present bivention; 

FIG. SC shows example (2) of a data packet in the 

embodiment ol the present frrventicn; 

FIG. 6D shows an example of a control packet in 

the embodiment of the present Invention; 

RQ. 7A explains the principle of a packet link in the 

embedment of the present invention, and shows a 

video frame; 

RG. 7B explains the principle of a packet link in the 
embodiment of the present invention, and shows 
encoded data of one frame; 
FIG. 7C explains the principle of a packet link In the 
embocfiment of the present Invention* and discrimi- 
nation of the frame boundary; 
FIG. 8 Is a table tor explaining the layers of a packet 
type identifier in the embodiment of the present 
Inventton; 

FIG. 9 is a table for explaining a partially abandon- 
aWeflag and an absolute priority flag in the embod- 
iment of the present Invention; 
FIG. 10A is a table showing an example of aesiorv 
ment of packet type bentrflens (aseurt^g MPEGO in 
the embodiment of the present irrvenfon; 
FIG. 10B is a table showing an example of assign- 
ment of packet type identifiers (assuming MofJon- 
JPEG] in the embedment of the present invention; 
FK3L 1 1 A is a table showing a setting example of the 
packet priority order (packet tientaiere) in the case 
of a stream consisting of a single program In the 
embocfiment of the present invention; 
FK1 1 1B is a tsbte showing a setting example of the 
packet priority order (the correspondence table 
between the identifiers and priority levels) in the 
case of a stream consisting of a single program In 
the embodiment of the present invention; 
FIG. 11C Is a table showing a setting example of 
the packet priority order (explanation of packet 
processing) in fie case of a stream consisting of a 
sfne^e program In the embocSmanl of the present 
invention; 

FKl12Atea table showing a setting example of tie 
packet priorfty order (packet idenffliers) in the case 
of a stream made up of two programs in the embod- 
iment of the present invention; 
Rai2Bisatable6hov^aset^exarnp<eofthe 
packet priority order (the correspondence table 
between the identifiers and priority levels) in the 
case of a stream made up of two programs in the 
errexxfirnent of the present kiventiort 
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FK2L 12C to a table showing a setting example off 
the packet priority order (exportation off packet 
processing) In the case of a stream made up of two 
programs in the embccfment off the present inven- 
tion: 

FK3L 1 3 shows a setting example of an idemif iarpri- 
orfty correspondence taUe Including partially sban- 
ctonabteAfftabandonable flags in the embodiment off 
the present Invention; 

RQ. 14 shows an exanpto off a fltertng Information 
table in the embodiment of the present invention; 
FTQL 15 Illustrates the method of discriminating the 
frame boundary in a multiplexed stream using a link 
field In the embodiment of the present Invention; 
FIG. 1S to a flowchart lor explaining the outline off 
the processing of a stream shaping processing unit 
In the video transmission system In the embodi- 
ment of the present invention; 
FIG. 17 to e (tow chart showing the simple fOtertng 
discrimination processing in the entxxSment of the 
present Invention; 

FIG. 18 Is a table showing changes over time In 

packet transmission state off video (tela in the 

embodiment of the present Invention; 

FK3L 19isafkwc*artehowirxjtr>et^^ 

I nation processing inducing bit rate control In the 

ernbecfiment of the present invention; 

FIG. 20 to a flow chart showing the initialization 

processing In FIG. 19; 

FK3. 21 is a flow chart showing in detafl the cycle 
head proces si ng In F1GL 19; 
FIG. 22 to a table for explaining the transmission 
result confirmation processing of the previous cyde 
In the embodknerd of the present invention; 
FIG. 23 is a flow chart showing in detai the frame 
count processing in FK3L 19; 
FIG. 24 to allow chan showing in detail the discrim- 
ination code generation processing in FIGL 19; 
FIG. 25 to a table showing the code assignment of 
comparison arithmetic operation results upon gen- 
eration of the discrimination code In the embodi- 
ment of the present invention; 
FIG. 26 is a table showtag the correspondence 
between the discrimination codes (assigned codes) 
and the processing contents of the packet pro ce ss* 
Ing in the embodiment of the present invention; and 
FIG. 27 to a graph showing changes in bit rale as a 
result of thestream shaping processing of the video 
transmission system in the embodiment of the 
present Invention. 

The embodiment of the present invention wil be 
descrbed hereinafter with reference to the accompany- 
ing drawings. FIG. 1 to a block diagram for explaining 
the outline of a data transmission system of the present 
Invention* This embodiment win exemplify, aa the data 
transmission system, a video transmission system 
which transmits video data including audio data as an 



encoded packet stream. 
(Outline] 

6 The ouiilna of the video transmission system In this 
embodiment wil be described below with reference to 
the block diagram shown In FIG. 1. 

In this embodiment, a stream shaping processing 
unit 02 having a bit rate control function Is added to a 

to video transmission system that processes video 
streams encoded by MPEGC Motion -OPEQ, or the like. 

In a network used by a plurality of users, a sufficient 
network bandwidth (defined by, e.g., bps) required for 
distributing a video stream (encoded stream) cannot 

io always be assured. For this reason, even when the user 
wants to transmit a stream at a bit rate o1, ag., 6 Mops, 
If a sufficient network bandwidth cannot be assured, the 
stream must be distributed after its bH rate to reduced to, 
e.g„ 2 Ufepsbyfhebft rate control. Upon bit rate redue- 

20 tion, me quality of me decoded Image to preferably pre- 
vented from being Impaired as mush as possible. To 
achieve ft is, the video transmission system of this 
embodiment usee the following method. 

An encoded stream is packatized In units of aban- 

25 donatio encoded data, and is stored as a stored stream 
\n an external storage device 08. The stored stream has 
a stream structure suitable for bit rale control (transmis- 
siortfabanctoriment discrimination) by means of packet 
fitering. 

so The definition of the stream structure prescribes 
packetizing off a stream in units of encoded data, cycl- 
cai insertion of control packets indicating coding cycles 
Mo a stream, and a packet header format (packet Id en- 
alar also serving as a packet priority level). Since this 

ss stream structure or architecture includes no prescrip- 
tions about encoded data itself, it can be easily 
expanded from existing moving image cocfng schemes 
such as MPEG. Motion-JPEG, and the Cka. 

The stream shaping processing unit 02 performs 

40 stream shaping processing (bit rate control method) for 
an encoded stream, which is Input from an Input unit 00 
and etored In the external storage unit oa by packet «• 
tertng using packet identifiers, which ateo serve as 
packet priority levels and are added to the headers off 

45 the Individual packets. Thereafter, the unit 02 transmits 
a stream onto the network via a transmission unit OA 

Upon executing the stream shaping processing, the 
stream shaping proc essi ng unit 02 determines in 
accordance with the priority assigned to each packet 

so identifier registered In a table 03 if each packet is to be 
abandoned or transmitted. 

Tb aSow bit rate control by rujrneiic^veAie designa- 
tion, the stream shaping processing urtft 02 preferen- 
tially transmits packets in the order from those having 

ss higher priority levels wtthln the range of the available 
network bandwidth on the basis off the correspondence 
between the packet identifiers and priority levels regis- 
tered bi the table 03» and Increases the number off pack- 
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ets to be transmitted within the allowable range. As for 
packets having identical priority within a single cycle, it 
Is determined thai the leading one of these packets has 
higher priority. 

The bit rate control method wfl) be brief ly described 
dgjow. pacMeis raving prion r/ nigner man max wmcn 
serves as a boundary upon discrlminaflng H a packet Is 
tranernrtteoy&bandcned are unconditionally transmitted, 
and packets raring priority lower than that at the dis- 
crimination boundary are unconditionally abandoned 
As for packets having me same priority as mat at the 
discrimination boundary, the number of packets to be 
transmitted is gradually increased as the number of 
cycles Increases. When all the packets wWch have the 
same priority as mat at the discrmimBcflbc^Jaryand 
are present in a single cycle have been transmitted sev- 
eral cycles later, the priority of the disci Irritation bound* 
aiy is lowered to aOow transmission of more packets. 
Upon repeating such processing, the bit rate gradually 
approaches the designated Wt rate. When the bit rate is 
about to exceed the descried btt rate, Increasing of 
the number of packets to be transmitted Is stopped, and 
thereafter, 8 nearly steady state is attained. When the 
correspondence between the packet identifiers and pri- 
ority levels registered in the table 03 te allowed to be 
arbitrarily changed, the number of packets to be prefer- 
erttJeJly output can be changed. 

Note that the encoded stream to be transmitted 
need not always be stored in the external storage unit 
OS, and an encoded stream which is packefized accord- 
ing to the stream structure prescribed In this embed- 
ment may be cfirectly input Also, the designated bit rate 
can be dynamically changed during stream transmis- 
sion. 

[System Arrangement] 

( Video Transmission System) 

The arrangement of the video transmission system 
in thb embodiment wfO be explained below. FIG. 2 Is a 
block diagram showing the arrangement of the video 
transmission system In this ernbcx&nant The video 
transmission system in this embodiment comprises an 
Input unit 10, a stream shapfog processing unit 12. a 
transmission unit 14, and a stream information acquisi- 
tion unit 16. 

The l^3ut unit 10 reads an encoded stream from an 
external storage unit 18 comprising, e.g.. a hard disk 
device, and outputs II to the stream shaping processing 
unit 12 in unrts of packets. The stream read rate is deter- 
mined with reference to the bit rate of the encoded 
stream (Input stream) supplied from the stream Informa- 
tion acquisition unit 1 6. The input unit 10 is assumed to 
have a buffer with a sufficiently large capacity Note that 
the input unit 10 may ctrectfy receive an encoded 
stream from an encoder (a stream encoder 34 to be 
described later). 



The stream shaping processing unit 12 reads the 
encoded stream in units of packets via the input unit 10, 
end determines using Information added to each packet 
header including packet priority (the packet format will 

5 bedescrbed to detail later) If each packet is to be trans- 
mfitedfebandorted. The unit 12 sends to the transmis- 
sion unit 1 4 only the packets which are determined to be 
transmitted. The stream shaping processing unfl 12 has 
not onfy a function of executing simple packet filtering 

tp processing based on the packet priority, but also a func- 
tion of adjusting the bit rate of the output stream not to 
exceed the designs! ed target faff rate: 

Hie transmission una 14 holds the packets 
received from the stream shaping processing unit 12 in 

15 a buffer, and transmits them to a receiving system when 
the held packets have reached an app r op riate data vol' 
ume. When a target bH rate after stream shaping 
processing is designated, the transmission unit 14 exe- 
cutes transmission processing while maintaining the 

so designated rata. At this time, coding cycle information 
may be used. On the other hand, the unit 14 also has a 
function of outputttng a request tor lowering the target 
eft rate lo the stream shaping processing unit 12 when 
the network load Is heavy and the designated transmfs- 

£5 eion rate can no longer be maintained. 

The stream Information acquisition unit 1B reads 
information associated with packet identifiers (to be 
descried in detaa later) which include packet priority 
Jewels and are stored In the external storage unit 16 In 

so stream coding processing, and suppfee information 
necessary tor stream shaping processing to the stream 
shaping processing unit 12 on the basis of the read 
Information. The stream information acquisition unit 16 
Informs the Input unfl 10 of the bit rate of the input 

35 stream. 

The stream shaping processing unit 12 will be 
desenbed in detail below. The stream shapvig process- 
ing unit 12 comprises a table generation section 20, a 
ffiering discrimination section 22. a header extraction 

40 section 24, a packet holding section 2£ artdafftering 
IntormatJon table 28, as shown In FIG. 2. 

The table generation section 20 generates a corre- 
spondence table that defines the correspondence 
between the packet identifiers and packet priority levels 

45 on the basis of Information associated with the packet 
Jdenta iers received from the stream information aoqiisl- 
Con unit 1 6, and saves it as the filtering information taWe 
28. ArternalivBly, toe section 20 receives a correspond- 
ence table in which the correspondence between the 

bo packet Identifiers and pacta* priority levels has already 
been registered, and saves it as the filtering Information 
table 28. Trie«tering information table 28 includes infor- 
mation associated wfth control packets, end a target bit 
rate (strictly speaking, the maximum number of bytes 

55 tost can be transmitted during one ceding cycle). Hie 
table generation section 20 can change the contents of 
the f Raring Information table 28 to accordance wfth an 
external instruction, Lo,, the users designation 
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(OMOFF designation of the stream shaping function, 
priority designation, and bit rate designation (required 
bit rata)). If there is no user* Instruction associated with 
the priority table, the table Generation section 20 gener- 
al as a default filtering Information table using Informa- 
tion associated with the pocket dentifierB atone. 

The target bit rata can be dynarrtcaly changed dur- 
ing stream transmission. Assume that the target bit rate 
value Is externally Input However. If the system has a 
means lor monitoring the available network resource 
(banco, the bft rate may be dynamically designated 
using this means. On the other hand, when the trans- 
mission unit 1 4 can detect the back pressure of the net* 
work, H may have a function of transmitting a target btt 
rate change request (rate adjustment command) to the 
stream shaping processing unit 12. 

The header extraction section 24 copies header 
portions (header inform a tion) from the packets received 
from the Input unit 10, and sends them to the filtering 
discrimination section 22. Also, the section 24 sends the 
packets (Including the headers) to the packet holding 
section 28. 

The packet hofcfing section 26 tenpomrify hods the 
packets received from the header extraction section 24* 
and executes packet processing in accordance with 
transmtsskxVabBnuonmsnt determination messages 
from the fltering discrimination section 22. When a 
packet is to be transmitted, the section 26 trans mits h to 
the tr ansm is s ion unit 14 without deleting Its header. 
When there is no relay node (to be described later) hav- 
ing a stream shaping function In the network that con- 
nects a transmitting system and the receiving system, 
and the recetvfcg system requests an encoded stream 
from which packet headers are deleted, the section 26 
may delete the header portions and transfer ontydatato 
the transmission unft The header deletion function may 
be assigned to the header extraction unrt 24. 

The filtering discrimination section 22 obtains 
packet priority levels by looking up the filtering informa- 
tion tatfe 28 using me packet idenffiiers included in the 
header Information sent from the header extraction sec- 
tion 24 as keys, executes fatering discrimination on the 
basis of the obtained packet priority levels, and sends 
di scrimina ti o n results to the packet holding section 26. 
The filtering discrimination determines transmte- 
sUxvabandonment In units of packets. If the bft rate is 
designated the number of packets to bo transmitted is 
gradually increased In the order from packets with high- 
est priority to packets with tower priority I €*e*s (with ref- 
erence to cycle Informing packets (to be described 
later)) so as not to exceed Redesignated btt rata When 
the flftering function is OFF, the section 22 sends an 
packets to the transmission unit 14. 

(Video Communication System Including Video trans- 
mission System) 

The arrangement of the video communication sys- 



tem inducing the video transmission system shown in 
FIG. 2 win be descried below. FK3L 3 is a btock diagram 
showing the arrangement of the video communication 
system in this embodiment As shown In FIG. 3, the 

s video communication system Is bulK by a transmitting 
system 30. a network 31, and a receiving system 32. 
The transmitting system 30 Includes the video transmis- 
sion system shown In FIG. 2. 

The transmitting system 30 comprises a stream 

io coding unit 34. the external storage unft 18. and a video 
transmission unit 36. 

The stream codng unit 34 comprises a video/audio 
coding unit 40 end a packet processing unit 42. 

The video/audio coding unit 40 performs coding 

is processing for an input video signal and outputs 
encoded data to the packet processing unft 42. In this 
embodiment, the type of cooing scheme Is not particu- 
larly EmitedL For example, moving image coding such as 
MPEG, MotiorvJPEG, H.261. end the like Is used. Note 

so that the encoded data has a structure suitabie tor packet 
processi n g. 

The packet processing unit 42 executes processing 
for packetizing the encoded data output from the 
video/audio coding unit 40 In correspondence with the 

as prescribed stream structure (to be descnoed in detail 
later) unique to this embodment and generates an 
encoded packet stream. The packet processing in the 
packet processing unit 42 absorbs differences of vari- 
ous coding schemes used In the videokudto coding urrft 

so 40. On the other hand, control packets are Inserted into 
the stream In the stream coding unit 34. 

Note that a packet is the basic unit of packet filtering 
in the stream shaping processing unit 12. and the 
header portion (header Information) of each packet has 

as a packet type kJerrtifier added thereto. The packet type 
identifier is an Identifier (refloating the type of packet 
and can be hierarchically set Note that information 
a s so ciated with the correspondence between the 
packet type identifier and the packet type (video, audio, 

40 or the Eke) indicating the contents Of actual encoded 
data may be saved as another fQe or as a control packet 
at the head of the stream. 

The encoded stream output from the stream ooding 
unit 34 is stored in the external storage unit 18 or is 

45 directly supplied to the video transmission unit 36. fo 
this case, the video/audio coding unit 40 that performs 
cooing processing and the packet processing unit 42 
that performs packet processing are separated, and 
their processing operations are executed sequentialy 

so but may be executed at the same time. On the other 
hand, the stream cooing unft 34 may have a function of 
multiplexing a pturafity of video signals to fcrm a single 
stream. 

The video transmission unit 36 corresponds to the 
55 video transmission system shown bi FIGL 2. The video 
transmission unit 36 not only generates the correspond- 
ence table between the packet Identifiers and packet 
priority levels using only stream Information acquired by 
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tho stream Information ooquisftjon unit 16, but also con 
reflect externaDy input user's designations (ON/OFF 
duUyiiafion of the dream shaping function! priority des- 
ignation, and bit rate designation). 

A setting unQ 38 is arranged to allow the user to « 
externaDy input his or her designation (information for 
generating the fflterfog information table} Id the video 
transmission unit 36. The setting unit 38 has an appro* 
prlate GUI (QraphlcaJ User Interface), and can present 
to the user Information associated with the packet type to 
identifiers indicating priority levels and the (to in the 
stream, and the actual packet types (video, audio, or the 
like) on, e.g.. a display device (not shown). When the 
setting unit 38 detects a desirjiatfon for changing the 
presented contents input by the user, ft informs the table rs 
generation section 20 in the stream shaping processing 
unit 12 of It Afeo, tho setting unit 38 receives the desig- 
nated number of bits (requested bit rate) of transmission 
data per sec, which indicates the available network 
bandwidth, and informs the table generation section 20 so 
of ft. 

Note that the setting unit 38 not only inputs user's 
designations in the transmitting system 30, but also can 
receive designations Input at the receiving system 32 
via the network and can reflect them in trie correspond- ss 
ence table (eg., can change the priority older or the 
Ifce) if it te connected to the receiving system 32 via the 
network. 

On the other hand, the receiving system 32 com- 
prises a reception unit 50 and a stream decoding unit 90 
52. 

The reception unrt 50 receives the encoded stream 
via the network 31 and sends H to the stream decoding 
unit 52. 

The stream decoding unit 52 eomprisee a packet as 
disassembly processing unit 60 and a video/audio 
decoding processing unit 62. 

The packet disassembly processing unrt 60 
removes the packet headers added tor the stream shap- 
ing processing to reconstruct the pure encoded stream, *o 
and sends It to the vtdeoVhudto decoding processing 
unit 62. 

The video/audio decorfng processing unit 62 
decod es the pure enco de d stream sent from the packet 
disassembly processing unit 60 to obtain video/audio 4s 
signals, and outputs the decoded signals. In this case, 
the packet disassembling processing in the packet de- 
assembly processing unit 60 and the deoodirig process- 
ing in the vfoecVauolo decoding processing unit 62 are 
separated, but may be e xecut e d concurrently. Afterna- 50 
livery, a (pure) encoded stream from which the packet 
hewers are removed may be reserved 

( Stream Relay Nod e With Stream Snap' ng Function > 

SB 

FIG. 4 shows the enartgemem of a relay node 70 
with the stream shaping function 

The relay node 70 b arranged on the route of the 



network 31 between the transmitting system 30 and the 
receiving system 32. The relay node 70 is used for 
absorbing different transmission rates of encoded 
streams between, eg., networks with different network 
bandwidth, and an arbitrary number of relay nodes can 
be set at arbitrary locations. 

As shown In FIG. 4, the relay node 70 comprises a 
communication interface unrt 71. a stream shaping 
pro cess i n g unit 72! a reception unit 74. a transmission 
unH 76, and a table reception unit 78. 

The communication interface unit 71 is an interface 
trial exchanges an encoded stream (andafiftering Infor- 
mation table) with the network 31 . "The unit 71 outputs 
the received encoded stream to the reception unit 74. 
and outputs the stream-shaped encoded stream from 
Vie transmission unit 76 onto the network 31 . 

The stream shaping processing unK 72 has the 
same functions osmose oi the stream shaping process- 
ing unfl 12 shown in FJQ. 2 (a detailed description 
thereof will be omitted). 

The reception unit 74 and trie transmission unit 78 
respectively have the same functions as those of the 
input unit 10 and the transmission unit 14 shown in FIG. 
2. and a datafled description thereof will be omitted. 

The table reception una 76 receives the fatering 
information table In which the correspondence between 
the packet identifiers and packet priority levels i s regis- 
tered from the corrmrtcation interface unit 71 via the 
rietworki and provides It to 9ie stream shaping process- 
ing unit 72. Note that the firterfng irtformalion table may 
be Inserted in a stream as a control packet 

When the above-mentioned relay node 70 is 
inserted between the transmitting system 30 and the 
receiving system 32, a plurality of stream shaping 
processing units are connected until the encoded 
packet stream reaches the receiving system 32. The 
stream shaping pro ces sing unit 12 in the video trans- 
mission unit 36, and the stream shaping processing unit 
72 in the relay node 70 respectively execute the stream 
shaping processing for the e nc oded stream by looking 
up the filtering information table Upon executing multi- 
cast cocTTnurdcalwra^ 

node has fBtering Information tables (and other parame- 
ters required for the bit rate control) corresponding in 
number to transmission nodes, and the bit rate control is 
done In correspondence with the available network 
bandwidth between the relay node and each transmis- 
stonnoda 

[Stream Structure] 

The structure of the e n co de d stream in this embod- 
iment wffl be described In detail below; The encoded 
stream in this embodiment has the foflowing faaturee. 

(1) rt is a packet stream that handles real-time data 

such as video data, audio data, end the tlka 

(Z) it rnuWpJexBs a plurality of types of data (video, 
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audio). 

(3) H can be either an Irtfrarrame encoded stream 
(eg.. Motion -JPEG) or an interframa predictive 
encoded stream (e.g., MPEG). 

(4) It Is premised of the assumption that data can be 
dropped In units of packets. 

(5) It defines the packet structure alone (Independ- 
ent fiom the coding scheme). 

(6) ft hdudes a packet type Wenflfler In *e header 
of each packet 

(7) It aJJows insertion of statistical information and 
control kitormaflon associated with the stream as a 
control packet 

(0) ft has control packets Indicating coding cycles 
cyd'eaDy inserted. 

The stream structure wiD be descrbed In detail 

The stream structure of this enfcodbnerrtlsusedfbr 
a fixed-length encoded stream of an arbitrary coding 
scheme (MPEQ. MctfornlPEG, H.281, and the like). 
Note that a stream must have a stream structure which 
is packefeed in units of abanoonabfe encoded data 
(e.g., video data tor one frame), and In which a header 
"wadded to each packet The header of each packet has 
a packet type identaier added thereto. This identifier 
also serves as packet priority, and stream shaping 
processing is attained by packet fBtermg using this prior- 
ity 

The packet filtering determines transit* saiorvaban- 
donxnBrrt In units of packets. Therefore, one packet must 
not include more than one encoded data unite (for 
example! one packet must not Include encoded data 
across two frames). 

Conversely, one encoded data unit can be divided 
Into a plurality of packets (for example, video data tor 
one frame is broken up into a plurality of packets). In this 
case, there must be an Indication that daarly shews ma 
data in the plurality of packets belong to a single 
encoded data unit For this purpose, a field indicating 
the link relationship wtth the previous and next packets 
is prepared in the packet header. Such field is prepared 
to facilitate the packet fttertnq processing. 

In the above example, a frame is exempted as the 
encoded data unit On the other hand, when one frame 
data is Hararchlcaily encoded and is divided Into a low- 
resolution cornponent and a hsgJwesoJution component 
mat compensates for the low resolution component 
these components are r es pec ti vely proc es s e d as indi- 
vidual encoded data units, 

The stream can include a control packet lhat has a 
special identrfier. This control packet is used for insert- 
ing statistical Mormation and control Information Into 
the stream. 

More specifically, as shown In FIG. 5, control pack- 
ets indicating coding cycles are inserted, and filtering 
processing and bfl rate control are done wtth reference 
to these packets* Note that the coding cycle kxfeates 



the cycle of coding processing Eke a OOP (Group of 
Picture) structure in MPEQ. In me case of Mction- 
JPEG, the coding cycle is defined with reference to the 
frame rate. The control packet can also be used for 

s informing the relay node or receiving node of the data 
size abandoned by fitering, and the like. 

This stream structure prescribes only the packet 
processing method and the information to be added as 
the header, if an encoder and decoder are designed to 

io keep such prescriptions, the bfl rate control method ki 
this embcdkrtertt can be applied to arbitrary coding 

[Packet Header] 

15 

The header information to be added to each packet 
in the encoded stream bi this embodiment will be 
described below. The packet header in this embodiment 
has the following features. 

so 

(1) ft has a synchronization bft field, link field, 
packet type Identifier field, and packet length field 
(the Bnk field and packet length field are not oxfis- 
pensatxe). 

25 (2) It afiows identification of control packets and 
stream oala packets by checking a specific bft In the 
packet type identrfier. 

0) ft anew* confirmation of the link relatkmsh^p with 
the previous and next packets (having the same 
ao packet type identifier) by checking the link field. 

(4) Its packet type Hentffer field can be divided Into 
a plurality of subfielda to have a hierarchical struc- 
ture. 

(5) lis packet type Iderrliier also serves as packet 
as priority. 

(6) Its packet type identifier field can have a flag that 
distinguishes absolute priority and variable priority 
from each other. 

(7) Its packet type iderttifier field can have a flag that 
40 stingiishes a partially abandonabie packet and a 

partially unabandonaUe packet from each other. 

The packet header will be described In detail below 
with reference to the accompanying drawings. 

45 As shown In FK1 6A. the packet header is made up 
of a synchronization bft field. (Ink field, packet type iden- 
tifier field, and packet length f iekl Each held size is not 
prescribed. However, fie packet header size preferably 
is a fixed length (for attaining high-speed processing). If 

so the paytoad size of the packet Is a fixed length, the 
packet length field is not necessary. 

(a) The synchronization bit field Is added for the 
purpose of detecting the packet header from the 
65 stream, and a fboed bft sequence Is written In mis 
field to aDow easy header detection. The packet 
type Idenffilar indicates the type of data Included h 
the paytoad of the packet 
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(b) The link field indicates the link state in each 
packet with respect to the previous and next pack- 
ets when encoded data for one video frame b 
divided into a plurality of packets, as shown in 
FIQa 7A to 7C Shoe the flnkfiefcl Is added, If the 
previous packet is abandoned by fltering process* 
Ingand the packet of Interest has a link relationship 
wfth the previous packet the packet of Interest can 
be automatically determined as the packet to be 
abandoned. The Ink field b prepared to fad) Kate 
packet filtering processing. When packets having 
the came pacta! type Identifiers continue, the link 
field serves as a boundary of adjacent packets. The 
Mowing example expresses the link relationship 
wflh the previous and next packets using 2 bits: 

(DO): Independent packet (no link relationship) 
(01): having a link with the next packet having 
the same identifier 

(11): having a link with the previous and next 
packets having the same Identifier 
(10): having a link with the next packet having 
the same identifier 



In a stream to which a plurality of types of data 
are multiplexed, packets having different packet 
type identifiers are mixed. For Ws reason, a packet 
having a cffferent packet type Identifier may be 
inserted between packets that are linked, 
(c) The packet length field Indfcates the payioad 
length (or packet length) of the packet The stream 
shaping processing loote up the packet header 
atone, but does not look up any data In the payioad. 
For this reason, the packet length information is 
added to allow easy detection of the packet header 
in the stream. 

The payioad that follows the packet header 
incfajdee one encoded data unit or its portion The 
encoded data unit indicates a data unit (aa. data 
for one vtd so frame) which does not have a serious 
Influence on decoding pr o c es s i ng In a decoder (Le., 
does not seriously deteriorate image quality) If It Is 
dropped from the stream. Note that data units may 
have a priority order lite I, R and B pictures (Intra- 
Picture, PrecBcfive-Plcture, Bidirectional fy Precfic- 
tive-Ficture)lnMPEQ. 

(cQ The packet type Identifier Is used for cfiscrknlnat- 
ing the packet type. The most eiQnflicant bit in the 
packet type Identifier field serves es a control flag 
tor distinguishing a control packet and a data packet 
from each other. 

The packet type identifier also serves as packet pri- 
ority, The packet type kfertfler also serving as packet 
priority" has the following meaning. The packet type 
identifier b added upon stream cooing. For this reason, 
if "IdenSier - priority* Is permanently dstBrrnfcsd, the 



userta demand (ag , lor changing the priority order of 
video and audio data) can no longer be reflected In 
view of this proof era the packet type Identifier serves as 
oetaurt pecaet pnomy. 

s The packet type Identifier field can be divided Into a 
plurality of subffelcte to have a hierarchical structure, as 
shown In RGS. 6B, 6C» and 6DL 

m order to explain the hierarchical structure of the 
packet type Uentfler, an MPEG stream In which two 

io program s as sets of video data and audo data, and 
other additional data are multiplexed wiO be examined 
below. In this case, as shown in FIGL 8, the packet type 
identiner has three layers "p no gram ". "data type", and 
"picture type". 

19 For example, when program 1 atone is extracted 
from this multiplexed stream and is transmitted, packet 
olstr&xjtkxvBbandonrnent can be determined by looking 
up only the packet type identfi er in the first layer. On the 
other hand, when P and B pictures are to be dropped In 

20 an MPEG video stream, the packet type identifiers must 
be checked up to the third layers. Such hierarchical 
st r uctu r e of the Identifiers Is also effective in terms of 
data management. 

When a hierarchical structure Is provided to the 

£9 packet type identifier field, bits assigned to this field are 
cfstributed to the IndMoual layers. In this case, the bits 
are tfstnbuted eo that the subfield inducing upper bits 
serves as an upper layer. As for assignment of the 
number of an identifier, a smaller value is assigned to a 

so packet type with higher priority in princple- This e Co 
allow use of the Identifier as (default) priority even when 
the boundary between the individual layers of the iden- 
tifier field or the cor re sp o ndence between the identifier 
and the actual datatype cannot be detected. 

as The MPEG stream ts an toterframe predictive 
encoded stream, and has three different picture types, 
le., I, P, and B. In order to decode P and B pictures, 
decoded data of I and P pictures are required. More 
specifically the MPEG stream has absolute priority of I, 

40 P. and B pictures in me order named Upon packet fiter- 
Ing, the quality of the decoded Image considerably suf- 
fers unless Otis priority is kept lb prevent this, a flag 
Indicating absolute priority (absolute priority flag) Is pre- 
pared at the least significant bit of each identifier sub* 

4S ftekt Hthte flag Is T, H Indicates that the Identifier In this 
layer has absolute priority. 

m this case, streams of aucfio data and user data 
are assumed to be not hierarchically encode d . For this 
reason, rf a packet in the middle of the stream is aban- 

so ctaned, flia quality deteriorates majkadfy, or data trans- 
mftted after the packet is abandoned becomes 
nonsensical. Such stream is preferably subjected to 
processing tor determining whsthsraO packets are to be 
tra nsmitte d without allowing any packet abandonment 

ss crararottobefrartsri-rttBdrt 

(partially a b a n do n ab ie flag) Indicating if the stream is 
partialy abandoned b prepared at the second lower 
b& of each kfenfller subfield. If this flag is 1\ ft Irrfl- 
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catee that the stream te partially abandonaMe. 

FIG. 9 Is a table tor explaining the partfely aban- 
donable flag and the absolute priority flag. These two 
f lace are not indispensable, tf a sufficient sire cannot be 
assured lor the kJartffler ftaW. a table that defines the 
correspondence among the identifier subfieJds, partially 
abandonable flag, and absolute priority flag may be pre- 
pared wrthout adrjng these flags to the identifier field. 

FIG. 10A shows an example of assignment of 
parts! type Identifiers while assuming an MPEG 
stream, and FIG. 108 shows an exarnpie of assignment 
of packet type kJentSlets while assuming a Motion- 
JPEG stream. In FIGS. 1 0A and 108, each packet type 
Identifier fiek) has two layers, L&« data type and frame 
type (equivalent to picture type), 4 bite are assigned to 
each layer, and tie absolute priority flag and partially 
abandonable flag are added. 

"System" shown in the column of frame type" In 
HQ. 1QA Indicates a GOP header or the Ito In a layer 
higher than the pJctwe header of an MPEG video 
stream. In this example, the Idenffiler of the second 
layer has 4 bits, the teas! significant bit and the second 
lower bit of which are respectively the partially abandon- 
able flag and the absolute priority flag. Thus, using the 
upper 2 bits, the identifiers of system, I. P, and B pic- 
tures are assigned. In this case, In order to allow use of 
these identifiers as default priority levels, higher priority 
is assigned to an identifier having higher Importance in 
the ceding processing (I.e., the identifier assumes a 
smaller value). 

On the other hand, stoce there is no absolute prior- 
ity order among video, audio, and user data shown in 
FIGS. 10A and 10a the least significant bft of the first 
layer is set at "0", and Identifiers are assigned to three 
data types using the upper 2 bits. The priority order of 
tros portion can be changed by the user. As for audo 
and user data, the partialy abando na ble flag is turned 
off ("0"). and as for video data, the flag Is fumed on C 1 "). 

Since Mocon-JPEG used in FIG. 10B is ■rttrafrarne 
r^ng,tfwlrxirviovaJf 

clffcafly. since the frame types of video data need not be 
distinguished from each other, the identifier in the sec* 
end layer Is not necessary. However, this also means 
that there is no lead for determining the frame to be 
abandoned first lb solve mis problem, hi FIG. 10& 
kfentf lers are assigned to even and odd frames (but the 
frames do not have any absolute priority). In thra man- 
ner, the frame abandonment older may be determined 
upon coding using the identifiers. 

Note mat turning off the partially abandonable flag 
does not means that packet abandonment is absolutely 
inhibited. Especially, when a rridole priority level is set 
for partialy unabejidonefaie data, this data wfB be par- 
tially abandoned with high possfoffity. tn order to avoid 
this, the Hpjiest or lowest priority level Is assigned to 
partialy unabandonable data. 



[Packet Priority] 

The packet priority win be explained below. The 
packet priority in this entoocSment has the following tea- 
0 tures, 

(1) It directly uses the packet Identifier. 

(2) It can be changed in correspondence with the 
us art demand. However, if 1he absolute priority flag 

io and partialy abandonable flag are used, they must 
be taken into consideration. 

(3) It assumes preparation of a correspondence 
table between the packet idemJlers and packet pri- 
ority levels. 

1$ (4) it includes a level that indcates "always trans- 
mif and a level that indicates "always eJbandon". 

The packet priority In this embodiment will be 
explained In detail below. 
20 The packet identifier can be used as packet priority 
without any modrffcations, but the packet priority order 
may be changed prior to stream transmission. In order 
to change the priority order, the relationship between 
the packet Identifiers Included In the stream and the 
25 attributes (video, aufio, and the lice) of the packets 
must be presented In the form that can be understood 
by the user. The information (stream irtforrnation) for this 
purpose is saved in the form of a fie independent from 
the encoded stream or In the form of a control packet 
90 added to the head of the stream. 

In order to allow the user to set the priority order, 
the means for presenting the identiiere and their con* 
tents (setting unit 38), and means (table generation sec- 
Hon 20) for changing the set priority order are prepared. 
ss tn this case, the priority order of portions that have fre 
absolute priority order (determined depending on the 
coding scheme) is inhibited from being changed. 

Even padojts with absolute priority may be set at 
"always transmit" priority levels in the order from those 
40 with higher priority levels or may be set at "always aban- 
don" priority levels In the order from packets with lower 
priority. 

The method of setting the packet priority order wf] 
be explained below. 
is The packet type Identifiers Meats the priority 
order of packets but assume discrete values (since fitey 
have a hierarchical structure). The user may change the 
priority order in some cases. Therefore, a correspond- 
ence table between the packet type identifiers and 
so packet priority levels Isprepared to determine an Identi- 
fier which has priority higher (lower) by one level than 
that of a given identifier. 

FIGS. 11Ato 11C show a setting example of tiie 
packet priority order of a stream consisting of a single 
S3 program, and FX3S, 12A to 12C chow a setting example 
of the priority order of a stream inducing two programs. 
Note that the program Indicates a stream made up of a 
pturanty of video, audio, end usar data. FIGS. 11B and 
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12B shew examples of the correspondence table 
between the packet type identifier and pricrrty levels. A 
priority setting example D in FK3L 12B wll be explained 
beta* 

As can be seen from FIGa 1 2A to 12C, this stream 8 
includes twopnsgrams. each of whfeh includes an audo 
stream Since program 2 is set at priority lower than pro- 
oram i, program 1 is transmitted first and thereafter, 
packets belonging to program 2 ere gradually transmrt- 
ted If there Is a margki. ti program 2. an audio packet to 
has highest priority, and is set as a partially inabarxton- 
able packet (see HQ. 12A). tn this manner, soma pack- 
ets are partially unabandonable although they have 
middle priority levels. Whether or not a packet Is par- 
tially a b a n dona We can be confirmed by checking the « 
epeeffic oil in the identifier. Also, a flag indicating H a 
packet Is partially abexxfor^eAjnabandortatxe maybe 
prepared in a co r r e spondence table (iderrflf ier-priority 
conespondence table), as shown In RGL 13. 

Note that the packet priority levels always assume so 
successive values. However, "0x00* that means 'always 
transmrT and "OxFF" that means "always abandon" 
need not be successive values. 

[Information Required for Stream Shaping] ss 

The information required for the stream shaping 
processing wil be described in dotal below. 

( Information Saved Upon Coding) so 

Each information to be listed below is saved 
together with the encoded stream In coding processkig. 
and is required for the bit rate control upon stream shap- 
ing. Such information (stream information) is acquired ss 
by the stream ^formation aco^jisiDon unit 16 prior to the 
start of stream transnussion. 

(1) Brt Rate of Encoded Stream 

40 

The bit rate of an original encoded stream Is 
informed. The informed bit rate is used as the reading 
rate of the encoded stream from the externa] storage 
unit 18. 

45 

(2) Hierarchical Structure and Priority of Packet T>pe 
Identifiers Included in Stream 

The correspondence table between all the packet 
type Identifiers Included In the stream and their so 
attrfoutes (video, audio, and the eke, i.e.. information the 
data type of which the user can Discriminate) is saved 
upon coding. When each packet identifier field has a 
hierarchical structure, the numbers of cits of the incSvid- 
ual layers fire also grven. These data are required when ss 
the user sets packet priority. 



(3) Time of One Ceding Cycle 

It is premised on the assumption that the coding 
processing of a video stream has cyd icity Control pack- 
ets that Indicate coding cycles are Inserted Into a 
stream to teach given time intervals. This value is used 
for obtaining the number of bytes to be transmitted per 
coding cycle For example, when the GOP (Group of 
Picture) In the MPEG Is used as one coding cycle, the 
reproduction fl r arar r sss tan) time of data for one OOP Is 
given 

(4) Packet Header Structure 

When the synchronization bit field, identifier fieUL 
and the like of the packet header have variable sizes, 
information associated wRh the packet structure is given 
(such Information is not necessary if the heater struc- 
ture is fixed). 

(5) Information Associated with Control Packet 

Information indicating the correspondence between 
the packet ktentMiers and the meartfaigs of control pack- 
ets is given (preserved). In this case, assume that cycle 
Informing packets and a stream and packet indicating 
the end of the stream are inserted. Also, a packet for 
informing the receiving system of the parameters and 
flie like of the transmitting system can be prepared. 

< Parameters Acquired Before Beginning of Stream 
Transmission) 

(1) ON/OFF and Mode of Packet Filtering Function 

When a stream b to be directly transmitted, the 
tering function is allowed to be turned off. There are two 
fDtBring modes. La, simple filtering processing, and fil- 
tering processing with the bit rate control. 

£2) Designated Bft Rata 

Upon exacuflno the packet filtering processing with 
the bit rate control, the bit rate of a stream after process- 
trig b designated. trtskSe the system, the designated bft 
rate is held In the form of the number of transmittabie 
bytea (Bpc: Bytee per cycle) per coding cycle. 

(3) Information Associated with Changes in Racket Pri- 
ority 

When the user changes packet priority, this infor- 
mation is acquired. FkiaOy, this Information Is saved as 
the filtering i nfo rm al: on table 28, La, the correspond- 
enrotHbtebetweenthaidertr^ 
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(Contents of Filtering Information Table: 3 Types) 

(1) Correspondence Table Between Packet Identifiers 
and Racket Priority Levels 

The correspondence table between the packet 
Identifiers and packet priority levels Is generated ki con- 
sideration of the default priority levels of the packet iden- 
tifiers and the user's setting. HQ. 14 shows an example 
of the filtering Information table, bi this example* the pri- 
ority levels assume values ranging !rorn 0x00 to OxFF 
expressed by 8 bits, As the value te smaller, the priority 
Is higher. 0x00 indicat e s a packet to be "always trans- 
mitted", and OxFF kxflcates a packet to be "always 
aJbarxionecr (priority need not always be expressed by 
8 bits). 

Not a that packets having afferent identifiers but the 
same priority must not be present and the packet prior- 
ity levels must assume successive values. 

(2) Number of Iraremlttable Bytes per Cycle (reqJ3pc) 

This value indicates the number of data bytes that 
can be transmitted per cycle (the Interval between adja- 
cent control packets indicating the cooing cycles). Nor- 
mally, the required bft rate (avalable network 
bandwidth) is given as the number off bits per second 
(bps: bits per second) of transmission data, and the 
value obtained by converting this value Into Bpc (Bytes 
per cytfe) is held in the tabla 

Bpc can be changed In accordance wfth an externa} 
instruction during stream transmission. (If Bpc is 
changed to a smaller value, the discrimination boundary 
priority level (clscrimlnafion boundary level; to be 
descrfeed later) is initialized. H Bpc is changed to a 
larger value, the f tttertig discrimination processing con- 
tinues.) 

(3) Irrfcrmation Associated Witn Control Packet 

In Ms embodiment cycle informing packets and a 
stream end packet are used. The correspondence 
between the Identifiers and the attrtxites (cycle Infor- 
mation, stream end) of control packets is assumed to be 
determined In advance. 

< Data Held by Filtering DiscrUw'nafion Section 22) 

(1) Header Information of Packet to be Filtered 

The contents of the packet type identifier field, link 
field, and packet length fields are copied. The data in 
the packet length field Is used for counting the number 
of transmitted bytes. (When the data in the packet 
lenglhfietttndtetesthep 
added thereto) 



(2) Discrimination Boundary Level 
(boundaryjevel) 

This level cor respon ds to the packet priority level 
o serving as the discrimination boundary upon determin- 
ing whether a given packet is to be transmrtted or aban- 
doned. A packet with priority having a value smaller 
tan the discrimination boundary level (le., higher prior- 
ity) b determined to be Transmitted*. Conversely, a 
io packet wfth priority having a value larger than the ds- 
crMrtaUon boundary level (Le., lower priority) is deter- 
mined to be "abandon ecT. Packets wfth priority having 
the same varus as the cDscnrnination boundary level are 
subjected to Uttering discrimination on the basis of their 
13 packet sizes, the number of transmitted bytes, and the 
value Bpc 

Upon completion of packet discrimination for one 
cycle, if there is no result or record of frame abandon- 
ment the value of the Discrimination boundary level Is 
20 IrKrememed by one (to lower the olscriminatlon bound- 
ary level). 

Even when packets with priority having values 
smaller than that of the discrinrtination boundary level 
are transmitted, confirmation processing Is executed 
25 based on their packet ekes, the number of transmrtted 
bytes, and the value Bpc. 

(3) Number of Transmitted Bytes (now_Bpc) 

so The packet sizes transmitted within one coding 
cycle are summed up. This value Is reset for each cycle. 

(4) Frame Counter (IramejcounQ 

$s A frame indicates the encoded data unit (data 
abandonment unit). In the case of video data, the frame 
corresponds to data for one frame. In the case of audio 
data, there Is no idea of frame whan It Is not Herarchi- 
caity encoded. The frame counter counts the number of 

40 frames tat have the same priority es trie Recrimination 
boundary level present In one cycle. Data for one frame 
Is divided into a plurality of packets In some cases. 
Accordingly, upon counting the number of frames* It is 
checked by looking up the Onk field in each packet 

45 header If the packet corresponds to a frame head 
packet The frame counter Is reset for each cycle. FK3L 
15 shows the method of discriminating the frame 
boundary hi a nxrttlplexed stream using the Onk field. 

so (5) Nurrfcar cf ThinsmJseion-permitted Frames 
{jp^rmr|_tnirnes) 

This value Indicates the number of frames which 
are permitted to be transmitted during one cycle and 
65 have the same priority as the dscrtmi nation boundary 
level. After packets for one cycle have been transmitted, 
ff the number of transmitted bytes in practice has not 
reached the number of transnittafcle bytes, this value is 



13 



25 



EP0853 407A2 



26 



ircremented by one (to gradually increase the number 
of frames to bo tmnsirtttecJ during on* cycle). 

(6) Ab an do nm ent Result Rag 

The abandonment result nag includes a byte ever 
flag and a frame ever flag. Using these flags, whether or 
not the number of packets to be transmitted per cyde Is 
Jo be Increased is determined. 

(a) Byte Over Flag (byteejover Jlag) 

This flag indicates that there is an abandonment 
result of pactats having the same priority as the dis- 
crimination boundary level since the number of trans- 
mmable bytee would have been exceeded rf they were 
not abandoned. This flag is reset tor each cycle. When 
this flag is OFF, it means that there is no packet aban- 
donment result: H the flag is ON. ft means that tiers Is a 
packet abandonment result 

(b) Frame Orcr Flag (frajra_over_flag) 

This flag tndcates that there Is an abandonment 
result of packets having the same priority as the dis- 
crimination boundary level since the number of trans- 
rn^sion-permitted frar^ exceeded if 

they were not abandoned. This flag is reset for each 
cycle. 

(7) Fix Flag (fbrjlagj 

This flag Indicates that the number of packets to be 
transmitted is not Increased any more since the data 
volume transmitted per cyde has reached the number 
of transmtttabts bytes. 

The operation of this entxrfmentwiUbedescrtoed 
below. 

[Stream Shaping Processing] 

FIG. 16 a flow chart for explaining the out line of 
the processing of the dream shaping processing unit In 
trie video transmission system of the presert invention. 
The stream shaping processing wffl be described below 
with reference to tie flow chart srtwm In FIG. 18. 

The stream shaping processing unit 12 performs 
initialization processing, generation of the filtering Infor- 
mation table 28 by toe table generation section 20. and 
the Bke prior to stream transmission (step A1 ). Thereaf- 
ter, the stream shaping processing unit 12 starts the 
stream shaping processing simuftaneousfy with the 
beginning of transmission of an encoded stream from 
the input unit 10. 

The header extraction section 24 detects the packet 
headers from the encoded stream (step A2). The 
hoedar extraction section 24 extracts the packet header 
portions used In filtering Discrimination and transfers 



them as header i nformatio n to the filtering discrimina- 
tion secti on 22. Also* the section 24 transfers packets to 
the packet holding section 28 to store them In the sec- 
tion 26 (step A3). 

e The f Bering dlscrinrfnation section 22 executes fil- 
tering discrimination on the baste of information 
Included in the header Information, and Informs the 
packet holding section 26 of the Discrimination result 
(step M). As a result of discrimination, if It b deter- 

to mined thai the peckirtrfb 

packet holding section 26 outputs the held packet of 
Interest to the next transmission unit 14 (step AS). On 
the other hand, it H is determined that the packet of 
Interest Is to be ahandorted, the packet hokllng section 

70 2$ drops the pocket of interest 

rf the encoded stream input from the input unft 10 
has not reached Its stream end and packets stiU remain, 
the same processing is repeated for the subsequent 
packets, tf the stream end has been reached, the 

» processing ends (step A6). 

The stream sharing processing is executed by 
packet ratering. The packet filtering method Includes the 
following two methods. 

& (1) Simple Fatehng Discrimination Processing 

(2) Filtering Discrimination Processing with Bit Rate 
Control 

These tittering methods will be described Indetailin 
so turn below. 

(Simple Filtering DiscriminaSon) 

FIG. 17 is a flew chart showing the simple filtering 

as cfiscrirrinatton processing. In simple fBtering discrimina- 
tion, whether packets are to be transmitted or aban- 
doned is simply determined using the packet identifier 
values atone. In order to atbun this processing, the pri- 
ority level (0x00) indicating "always transmit- and the 

40 priority level (OxFF) indicating "always abandon" must 
be set In the correspondence table between the packet 
derrtfiers and packet priority leveia 

When filtering discrimination is started, the filtering 
cfiscrirTwiation section 22 acquires the packet identifier 

45 from each header (step B1). 

tf the acquired packet Identifier Indicates a control 
packet (Le.. a cycle informing packet or stream end 
pastel), the filtering dscriminatfon section 22 outputs a 
paotet t r an smission message to the packet hc^ftgsec- 

so Bon 26, and ends the tiering di sc ri mhation processing 
(may inhftxt the control packets from being transmitted 
to trie receiving system). 

On the other hand, If the acquired packet identifier 
indicates a date packet itself, the tittering discrirrinatjon 

55 section 22 looks up the filtering informafion table 28 
utfrxjtheMertmerasakeytoacq 
level conesponctng to the identifier (step B3). 

tf the acquired packet priority level Mtoates 
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"always tranartf (0x00), the tittering dBcrhwnation aeo 
ton 22 outputs a packet transmission message to the 
pad^hok&Ygsecik)n26arderdflthef9t8ring discrim- 
ination processing (step B5}. 

On the other hand. If the acquired packet priority 
level indicates "always abandon" (OxFF), the filtering 
discrimination section 22 outputs a packet abandon- 
ment message to the packet holding section 26 and 
ends the tittering discrimination processing (step B8). 

(Rtoing Discrimination Wrth Bit Rate Control) 

The packet fittering di sc r i min a tion melted with bit 
rate control will be explained below. 

FK3L 1 8 shows changes over time in packet trans- 
mission state of video data whfle taHng MPEG as an 
example. In the first cycle, oniy packets inducing I pic- 
ture data with highest priority are transmitted. From the 
next cycle, packets Inducing P picture data are gradu- 
ally added to the packets to be transmitted. Thereafter, 
when packets Inducting all I and P picture data are 
added tothe packets to be tr an srnrtted. andthedestg- 
nated brt rate has been reached, the number of packets 
1o be tr a n smi tted ceases to Increase* To attain tittering 
(fiacrirwnation. the following three different comparison 
discrimination processing operations are executed. 

(1) Cornparison between Packet Priority and Dts- 
crimination Boundary Level 

(2) Cornparison between Number of Transmitted 
Data Bytes and Number cf Maximum Transmittable 
Bytes 

(3) Comparison between Number of Transmission- 
permHted Frames and frame Counter 

tmrnecWsty after the beginning of stream transmis- 
sion, the cfscrirnmation boundary level is set at 
boundaryjevel - 0x00. Thte means that only packets 
with a priority value - 0x00 are transmitted, and other 
packats are abandoned Also, me number of transmis- 
sion-permitted frames Is set at permJL^Hmee - 0. The 
number of transrnission-peimHted frames indicates the 
number of frames that are permitted to bo transmitted 
within one cycle. 

When stream transmission Is started, and the fast 
cyde Informing packet Is detected, bourrfa/yjevel - 
0x01 and permftjrarnes - 1 are sat This means that 
transmission of pacto Is onrypanr^ 
a prforty value - 0x00, or if packets he*e a priority value 
-0x01 and belong to the frst frama 

In this case, packets with a priority value • 0x02 or 
larger are unconditionally abandoned. Upon completion 
of packet processing for one cyde, H the number of 
transmittal e bytes is not exc ee ded, perrnt_frames ■ 2 
Is set to permit to transmit packats with a priority value 
- 0x01 for two frames. 

After several cydes, if the number of transmfttable 
bytes Is not exceeded after al the paokete with me pri- 



ority value - 0x01 have been transmftted, the number of 
trajnsrrttssk>r>-pemiitted frames (pemtiUrames) is reset, 
and the discrimination boundary level is Incremented by 
one to set boundaryjevet • 0x02. 

a In the next eyelet packBtswfmpnorty 

and 0x01 are uiconcfitionally transmitted, and transmis- 
sionJabandonment of packats wfth a priority value - 
0x02 is determined m consider ati on of the number of 
transmission-permitted frames and the result of the 

io number of trartsrnfttad bytes. Atea packets with a prior- 
ity value - 0x03 or larger are gncontf&onaDy abai* 
oonoQ. 

As the number of cydes increases, the number ot 
transmitted packets gradually Increases, and soon 

is reaches the number of transmittal bytes per cyde. 
This means that the boundary level of packet fftering 
discrimination s determined. In the subsequent cydes, 
the discrimination boundary level remains the same, 
and the number of transmitted packats per cycle 

29 becomee nearly constant 

FIGL 19 te a flew chart showing the filtering discrim- 
ination prooessirtg with bit rate control This processing 
will be explained betow with reference to the flow chart 
hi FIG. m 

as Assume that irtmaiizafion processing (step D1) 
shown In the fkwcr^hna. 20 riasatreacV bean axe- 
cuted prior to the beginning of transmission. 

The fitering cfsoimination section 22 acquires the 
packet Identifier, Dnk Information, and packet length In 

so the individual fields from each packet header obtained 
torn the header extraction section 24 (step C1). The !l- 
taring olscrimtnation section 22 looks up the most signrf- 
Icant bit of the packet Identifier to check if the packet of 
tiler est la a control packet (step C2). 

sb if the packet of Inter est is a control p a cke t and a 
cyde packet (step C3). the fitering cfscrtmtnatkm sco- 
tion22 executes cyde start processing (step C4)- In this 
cyde start processing, the packet ebamionment result 
m the previous cyde is checked to update the discrirni- 

40 nation boundary level and the number of t r ansm i ssi on* 
permitted frames, and to reset various cycle variables. 

When the control packet is to be transmitted to the 
receiving system, the filtering domination section 22 
outputs a packet transmission message to the packet 

45 holding section 26 (step C5). updates the number of 
transmitted bytes (step C6). and then ends filtering cSs- 
Cffantnation (the control packet may not be transmitted)' 
On the other hand, if ma packet cf interest tea data 
packet (C3, the filtering dscrimination section 22 looks 

so up the fitering information table 28 using the header 
tferttifiereaakeytoeo^elhepacra 
responding to the identifier (step 07). 

The filtering dZscrirrshatfon section 22 executes 
frame count proc e ss in g (step C8; its processing proce- 
ss dure wm be descrtoed in data! later (RG. 23}). In Ms 
processing, the number cf tramee to which packets hav- 
ing the same priority as the discrimination boundary 
(svdbekxigbcountfribylccH^ 



15 



29 



EPOB53407A2 



SO 



Hnk fields m th8 packet headers. This Is to accurately 
count the number of frames even when data belonging 
to one frame is dMded Into a plurality of packets 

Subsequent the (tiering olecriminafion section 22 
generates a dtecrimlnalion code (step C9; this procees- 
ing procedure wfll be described in detafl later (FIG. 24)). 
Trie cSscrtnrirHtfon code summarizes the results of the 
three different ccrnparison discrimination processing 
operations, and Is used tor determMng packet trans- 
rntsslcfi/abandonrnertt 

The filtering drscrfminatJoo section 22 executes dis- 
crimination 1 (lor determining whether the packet Is to 
be transmfttedfabandoned) on the basis of ihe Discrimi- 
nation code (step C10). and then executes processing 
in correspondence with the dtesrsixnofibn result. 

If discrimination 1 determines that the packet is to 
be transmitted, the filtering discrimination section 22 
determines packet transmission, and outputs a packet 
transmission message to the packet holding section 28 
(step C15). Also, the section 22 updates the number of 
transmitted bytes (slap C16). 

At this time, the tutoring discrimination section 22 
executes discrimination 3 (step G17) to check if the 
number of transmitted bytes has exceeded the number 
of transmrttabis bytes, if ft is determined that the number 
of transmitted bytes has exceeded tha number of frans- 
rrtttable bytes, the section 22 sets the byte over flag » 
ON (step Clfl). 

With the above-mentioned processing, the filtering 
processing ends. 

On the other hand, if discrimination 1 determines 
that the packet ia to be abandoned (step C10). the filter- 
ing cSscrlmlnaJion section 22 determines packet aban- 
donment, and outputs a packet abandonment meesage 
to the packet hokfing section 26 (step C1 1). 

At this time, the tittering discrMnation section 22 
executes Recrimination 2 (step C12) to obtain the rea- 
son for packet abandonment and then executes 
processing corresponding to the obtained reason. 

More 6pecffica0y; when packet abandonment is 
determined because of taw priority, the tittering discrim- 
ination processing ends without any processing. On the 
other hand when packet abandonment Is determined 
because of an exceeded number of bytes, the byte over 
flag Is turned on (step C13). Also, when packet aban- 
donment Is determined because of an exceeded 
number of frames, the frame over flag is turned on {step 
C14). 

With the above-mentioned proc es s i ng, the filtering 
processing ends, 

FtQ.21 is a flow chart showing In detaJ the cyde 
head (start} processing (FIG. 19 (step C4)). 

The head of the cyde also means the and of the 
previous cycia Hence, in the cyde head processing, irt- 
tiallzsfion processing (for resetting various cyde varia- 
bles) at the head of the cyde is penormed (step E9). 
More specfncaTIy. the HSaibsaon processing at the 
head of the cycle resets the frame counter, the number 



of transmitted bytes, the number of abandoned bytes, 
the frame over flaa and the byte over flag- 
Together with the initialization processing at the 
heed of the cyde, the oSscrirrfnatSon boundary level and 

5 the number of transnrrisskxi-permtttod frames are 
updated and the fix flag state is changed as the trans- 
mission result confirmation processing of the previous 
cyde. FIG. 22 summarizes the transmission resuft con- 
firmation processing of tha previous cyde. 

70 More specifically. If the fix flag is OFF (step El), and 
it the byte over flag end the frame over flag ara respec- 
fivery OFF and ON (steps E2 and E5), skice the number 
of transmrttabJo bytes has not been reached yet the fiV* 
taring dtectMratlon section 22 increments the number 

15 cftranv^iobetre/rsmtted per cyde by one, (step E6). 
On the other hand, if the fix flag ie OFF (step E1 ), 
and If the byte over flag is ON (step E2), the filtering dis- 
crimination section 22 determines that the boundary 
(oval of packet transmfssion/aberxJonment Is deter- 
so mined, turns on the fix flag (step E3), and decrements 
the numbar ol transmtesto-permrttad frames by one to 
tall within tha afJowabfe range (step E4). 

Furthermore, if the fix flag is OFF (step E1). and if 
both tha frame ever flag and the byte over flag are OFF 

ss {step E5). since all the packets having the same priority 
as the discrimination boundary level have been trans- 
mitted and the number of transmrttabfe bytes has not 
been reached yet 8">e filtering discrimination section 22 
sets the number of transrrfssion-penTiftted frames for 

90 the next tower level at 1 (step E7). and increments the 
rjscrirrination boundary lave) value by one. I.e.. lowers 
the cCscrimirtafion boundary level (step E0). 

RG. 23 Is a flow chart showing in detail the frame 
ocunft procefi&hg (JIG. 19 (atspCS)). 

ss m the frame count processing, the filtering discrim- 
ination section 22 compares the priority level and the 
tfscrirrination boundary level (comparison 1) to check rf 
the packet of Interest has the same priority as tha dls- 
crirninatkxi boundary level (step F1). As a result, if the 

40 packet of interest has the same priority as the discrimi- 
nation boundary level, the Uttering discrimination sec- 
tion 22 counts the number of frames of toe packet. Ihe 
frame boundary Is confirmed based on the fink field in 
the packet header (steps F2 and F3). 

45 FK3, 24 Is a flow chart showing In detail the discrim- 
ination code generation process i ng (FIG. 19 (step C9)). 

The rjscriminafion code (6 bits) surnmarrzee the 
results of the three comparison arithmetic operations 
required for determining packet transrrrieskxVabarxJon* 

so ment 

(a) Comparison Arithmetic Operation 1 

TOority of a packer and >iorityc4thec08crfriwia- 
ss tion boundary ieveT are compared (step G2). With this 
comparison, packets whose transrrissiorVabartcxYV 
ment can be urxx x xfilio na Jfy determined are selected. 
The comparison arihmaae operation result Is saved In 
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the third and fourth lower bits of the Discrimination ood o 
(steps G3 and G4). 

(b) Compartcon Arithmetic Operation 2 

5 

"The frame number of a frame to which thepactet 
to be discriminated belongs" and "the number off trans* 
n^OfvpenTutted frames" are compared (step 05). S 
the number of transmtssio/t-pennfttBd frames has been 
reached, al packets havfrig the same priority as that of w 
the tf&crfmination boundary level are to be abandoned. 
The oofflpn Ison arithmetic operation result Is saved in 
the second lower bit of the discrimination code (step 
Q6). 

15 

(O Comparison Arithmetic Operafion 3 

The number of transmitted bytes if the packet to be 
dtecrtnhated Is assumed to be transmitted" and The 
number of transmittabte bytes per cyde" are compared 20 
(step Q7). If the number of transrrAtable bytes has been 
reached, this means that the number of packets to be 
transmitted per cyde can no longer be increased. The 
comparison arithmetic operation result ts saved In the 
least stgnflicant bit of the discrimination code (step G8). 23 

FIG. 25 shows the code assignment of the compar- 
ison arithmetic operation results upon generation of the 
dis criminati on code, and FK3. 26 shows the correspond- 
ence between the cfiscrtmination code (assigned code) 
and the pr oces s in g contents of the packet processing, so 

[Effect of Stream Shaping Processing] 

FIG. 27 shows changes In bft rate as a result of the 
stream shaping processing of the video transmission as 
system In this embodiment In FIGL 27, the bold Dne Indi- 
cates the stream after the shaping processing, the thin 
line Indicates the designated bit rate, the dotted I tie 
indicates the original encoded stream, and the alternate 
long and short dashed line indicates the stream sub- 49 
lasted to the tuB shaping pr ocessing. 

Immediately after the beginning of stream transmis- 
sion, since only packets wtth high priority are transmit- 
ted, the transmitted data volume "« lower than the target 
bit rate (designated bft rate). The runber of packets to 45 
be transmitted Is gradually increased In units of cooing 
cycles In consideration of packet priority, and the bit rate 
Is made to approach the target bft rate (see RQ. 18). 

During encoded stream transmission, if the bit rate 
Is changed to a value larger than the target bft rale des- 60 
ignated so tar, the number of packets to be transmitted 
is increased again from that in the transmission state so 
far, thereby mating the bft rate approach the new target 
bit rata 

Conversely when the target bit rate Is lowered, the 63 
initial state is restored, and the number of packets to be 
transmitted Is gradually harassed from packets with 
highest priority; thereby mating the bit rate approach 



the target bit rate. 

In this manner, an input stream is packetized In 
units of hierarchical data in do hierarchically encoded 
stream to obtain an encoded stream in which priority 
levels are added to the lrrfvtdua! packets, and the 
encoded stream is transmitted whOe executing bit rate 
control based on packet filtering In consideration of the 
available network bandwidth, packet priority, and user's 
demands. The bit rata control not only can set desig- 
nated levels prepared In advance, but also can arbitrary 
set the W rate of the stream to be transmitted, and can 
dynamically control the bft rate not to exceed the desig- 
nated bit rate. 

In the above embodiment, the encoded stream to 
be transmitted is the one in which video data, audio 
data, user data, and the Ike are multiplexed, and the bit 
rate control method of the video data portion has been 
mainly described. If audio data is hierarchically 
encoded, the audio data portion can be subjected to the 
bit rate control. Of course, the bit rate control can be 
made far other types of data. 

As described in detail above, according to the 
present invention, dynamic bit rate control can be real- 
bed according to the designated bit rate Independently 
of the moving image cooing scheme used. 

The present Invention is not limited to the above- 
descnoed ernboc fcnen t and can be rnodfted without 
departing from the spirit and scope of the invention. 

Claims 

1. A data transmission system comprising: 

Input means (00) for Inputting an encoded 
stream which is packetized in units of abandon- 
able data, and in which a header Including a 
packet identifier is added to each of packets; 
transmission means (04) tor transmitting the 
encoded stream input from said input means 
(00) onto an network; 

designation means 08) for designating a bft 
rate; end 

control means (02) lor controlling a bit rate 
upon transmission of the encoded stream by 
said transmission means (04) by abandoning a 
specific packet using packs* priority deter- 
mined on the basis of the packet Identifier 
Included h the header of each packet hi 
accordance with the bit rate designated by said 
designation means (38). 

2. A system Becoming 10 daim 1, wherein control 
packets Indicating codtig cycles era cycScafy 
inserted in the encoded stream, and said control 
means (02) determines transmission or abandon- 
ment of the specific packet not to exceed the bit rate 
designated by said designation means (38) whfle 
Increasing the number of packets to be transmitted 
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34 



bi units of cycles discriminated based on the control 
packets 

3- A data transmission system for transmitting an 
encoded stream vta a iietwork cojrjxtslng: 8 

stream coding means (34) for generating said 
encoded stream which is packeSzed in units of 
abandonable data, and In which a header 
Including a packet Identifier also serving as io 
packet priority Is added to each of packets; 
stream shaping processing means (12) for 
determining transmission or abandonment of 
each of the packets In the encoded stream 
generated try said stream cooing means (34) 75 
using the packet identifier included in the 
header of each packet In accordance with a 
designated bit rate; and 
transmission means (14) for transmitting the 
encoded stream made up of packets which are so 
determined to be transmitted by said stream 
shaping prooesstag means (12). 

4. A system accordkig to claim 3. wherein said desig- 
nation means (38) comprises means for dynamt* 20 
cafly changing the bit rata during transmission of 
the encoded stream by said transmission means. 

5. A system according to daim S, wherein sab stream 
shaping processing mesne (12) comprises: go 

table generation means (20) for generating a 
fiRnring information labia in which a corre- 
spondence between the packet dentfler 
included in the header of each packet of the as 
encoded stream generated by said stream cod- 
ing means 04), and packet priority is regis- 
tered; and 

filtering discrimination means (22) fordiscrirnh 
nafing transmission or abandonment of £&cft of 40 
the packets with reference to the filtering Infor- 
mation table generated by said table genera- 
tion means (20) on the basis of the packet 
Identifier included in the header extracted from 
each packet bi the e nc oded stream to be trans* 45 
fritted. 

6. A system according lo claim 5, wherein said table 
generation means (20) comprises setting means for 
settlng the correspondence between the packet so 
Identifier and packet priority by reflecting an exter- 
nally input designation. 

7. A system according to claim 3, wherein said stream 
coding moans (34) comprises generation means far ss 
generating the encoded stream by cycficaOy insert- 
ing control packets Indicating cooing cycles, and 
said stream shaping processfog means (12) deter- 



mines transmission or abanotonment of each packet 
not to exceed the bit rate designated by designation 
means (38) whDe Increasing the number of packets 
to be transmitted in units of cycles cfiscrirrunated 
dbsoq on me contra pacKets. 

8. A system according to claim 7, wherein said stream 
shaping processing means (12) comprises: 

holding means tor holding cSscrMraBon 
boundary priority to be compared with the pri- 
ority based on the packet identifier so as to 
determine transmission or abandonment of 
each packet; 

number of packets increasing means for gradu- 
ally increasing the number of packets having 
the same priority as the discrimination bound- 
ary priority in units of cycles discriminated 
based on the control packets by transmitting 
packets having priority higher than the dscrinv 
Ination boundary priority and abandoning pack- 
ets having priority lower than the discrimination 
boundary priority; 

priority changing means for lowering the cfis- 
crimmation boundary priority after all packets 
having the same priority as the discrimination 
boundary priority in one cycle ere transmitted; 
and 

means tor stopping Increases In the number of 
packets to be transmitted when the designated 
bit rale Is reached. 

9L A system according to dakn 3, wherein said stream 
cooing means (34) provides a hierarchical structure 
to the packet identifier included in the header, and 
generates the encoded stream which expresses 
packet priority based on contents of the individual 
layers of the hierarchical structure. 

10. A system according to daim 3» wherein said stream 
coring means (34) generates the encoded stream 
which allows to discriminate if the packet identifier 
Inducted fn the header has absolute packet prforfty 
or variable packet priority. 

11. A system according to daim 3. wh areln said stream 
cooing means (34) comprises means for c^neratirig 
an encoded stream hi which the packet Identifier 
included in the header can be set to indicate 
whether or not a packet fs abardonaUa 

12. A system acconSng to ciaim 3, wherein said stream 
coding means (34) comprises means for setting 
data indica£ng a link retaWtshp between previous 
and next packets In the header. 

13. A relay node which b arranged on a network route 
and is located between a data tr ansmi tting system 
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find & data receiving system, comprising: 

reception means (74) for receiving an encoded 
stream from said transmitting system; 
stream shaping processing means (72) tor 
determining transmission or abandonment of 
eadh of packets In said encoded stream which 
is packetized in unte of abandonable data. In 
which a header Including a packet Identifier 
also serving as packet priority is added to each 
of the packets, and which is transmitted from 
the data transmitting system, using priority 
based on the packet identifier included in the 
header of each of the packets In accordance 
with a designated bit rate; and 
transmission means (76) for transmitting the 
encoded stream made up of the packets which 
are determined to be transmitted by said 
stream shaping processing means to the data 
receiving system via the network. 

14* A data transmission method for transmitting an 
encoded stream via a network, comprising: 



mined not to exceed the designated bit rate wh3e 
increasing the number of packets to be transmitted 
In unite cf cydes discriminated based on His control 



10 



the input step of inputting an encoded stream ss 

which is padcBtizfld in units of abandonable 

data, and in which a header including a packet 

identifier is added to each of packets; 

the transmission step of transmitting the 

encoded stream input in the input step onto the so 

network; 

the designation step of designating a bit rate; 
and 

the control titep of oontrollkig s bit rate upon 
transmission of the encoded stream in the ss 
transmission step by abandoning a specflc 
packet using packet priority determined on the 
basis of the packet tierrtirler Included hi the 
header of each packet in accordance with the 
bit rate designated in the designation step. 

16> A data transmissfen method for transmitting an 
encoded stream via a network, comprising: 

transmitting said encoded stream which Is 45 
packBtizsd fen units of abandonable data and In 
which a header inducing a packet identifier is 
added to each of packets, whSs controlling a bit 
rate by transmitting or abandoning each of the 
packets using packet priority determined based so 
on fte packet identifier included in the header 
of each packet in units of packets in accord- 
ance with a dasigriated b9 rata. 

16. A method according to claim 15, wherein control 55 
packets indicating coding cycles are cycfcally 
inserted In the enco d ed stream, and transmission 
or abandonment of the specflks packet is deter- 
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